Towards automatic extraction of expressive elements from motion pictures : tempo

This paper proposes a unique computational approach to extraction of expressive elements of motion pictures for deriving high level semantics of stories portrayed, thus enabling better video annotation and interpretation systems. This approach, motivated and directed by the existing cinematic conventions known as film grammar, as a first step towards demonstrating its effectiveness, uses the attributes of motion and shot length to define and compute a novel measure of tempo of a movie. Tempo flow plots are defined and derived for four full-length movies and edge analysis is performed leading to the extraction of dramatic story sections and events signaled by their unique tempo. The results confirm tempo as a useful attribute in its own right and a promising component of semantic constructs such as tone or mood of a film

Taming the roving camera : a framework for creating professional home movies through narrative and cinesthetic elements

In this paper we present an application designed to improve the quality of amateur video production. The majority of home movie material is negatively impacted by two factors: lack of narrative content- "what to shoot?", and the absence or inappropriate use of cinesthetic elements for effective reinforcement of content - "how to shoot?". We leverage the age-old communicative powers of Story to answer the what For the second problem, the how, we turn to the corpus of aesthetic principles that constitute the film profession, which impact both technical and cinematic considerations for a given project

An embedded suggestive interface for making home videos

This paper describes a novel suggestive interface embedded in a smart camera prototype aimed at aiding home movie makers. We focus on the problem of generating shot capture suggestions suitable to the user\u27s filming context, intended audience and style, and formulate a novel aesthetic measure by which to judge proposed suggestions. Tight coupling between media and software allows the aesthetic measure to be sensitive to previous footage captures, including those taken without the system\u27s prompting, in a manner allowing flexible, end-to-end migration of the authoring task from user to machine. An approximate method is used to find timely, near-optimal solutions to the aesthetic measure. Qualitative evaluation in the form of a user study shows it to be a promising approach to the flexible home movie authoring context

Authoring multimedia authoring tools

Capturing devices, while continually becoming smaller and easier to use, have increased in capacity. They are also more connectable and interoperable, and their propensity to show up where they are least expected is surprising. Despite these advances, the video-capture experience is still frustrating. To achieve success, two issues need consideration. One is to determine what to capture and how, and how to handle the ensuring process required to transform the raw captured footage into a presentable multimedia artifact. Continual query on discourse theory, domain distinctives such as media aesthetics, human-computer interface issues, and multimedia data description standards is also important

Injection, detection and repair of aesthetics in home movies

This paper details the design of an algorithm for automatically manipulating the important aesthetic element of video, visual tempo. Automatic injection, detection and repair of such aesthetic elements, it is argued, is vital to the next generation of amateur multimedia authoring tools. We evaluate the performance of the algorithm on a battery of synthetic data and demonstrate its ability to return the visual tempo of the final media a considerable degree closer to the target signal. The novelty of this work lies chiefly in the systematic manipulation of this high level aesthetic element of video

Role of shot length in characterizing tempo and dramatic story sections in motion pictures

Motivated by existing cinematic conventions known as film grammar, we proposed a computational approach to determine tempo as a high-level movie content descriptor as well as means for deriving dramatic story sections and events occurring in movies. Movie tempo is extracted from two easily computed aspects in our approach: shot length and motion. Story sections and events are generally associated with changes in tempo, and are thus identified by edges located in the tempo function. In this paper, we analyze our initial founding of the tempo function on the basis that the distribution of both shot length and motion in movies is normal. Given that the distribution of shot length is approximately Weibull as confirmed in our experiments, we examine the impact of modelling and modifying the contributions of shot length to tempo. We derive an appropriate normalization function that faithfully encapsulates the role of shot length in tempo perception, and analyze the changes to the story sections identified in films.<br /

Novel approach to determining tempo and dramatic story sections in motion pictures

This paper presents an original computational approach to extraction of movie tempo for deriving story sections and events that convey high level semantics of stories portrayed in motion pictures, thus enabling better video annotation and interpretation systems. This approach, inspired by the existing cinematic conventions known as film grammar, uses the attributes of motion and shot length to define and compute a novel continuous measure of tempo of a movie. Tempo flow plots are derived for several full-length motion pictures and edge detection is performed to extract dramatic story sections and events occurring in the movie, underlined by their unique tempo. The results confirm reliable detection of actual distinct tempo changes and serve as useful index into the dramatic development and narration of the story in motion pictures

Automated film rhythm extraction for scene analysis

This paper examines film rhythm, an important expressive element in motion pictures, based on our ongoing study to exploit film grammar as a broad computational framework for the task of automated film and video understanding. Of the many, more or less elusive, narrative devices contributing to film rhythm, this paper discusses motion characteristics that form the basis of our analysis, and presents novel computational models for extracting rhythmic patterns induced through a perception of motion. In our rhythm model, motion behaviour is classified as being either nonexistent, fluid or staccato for a given shot. Shot neighbourhoods in movies are then grouped by proportional makeup of these motion behavioural classes to yield seven high-level rhythmic arrangements that prove to be adept at indicating likely scene content (e.g. dialogue or chase sequence) in our experiments. Underlying causes for this level of codification in our approach are postulated from film grammar, and are accompanied by detailed demonstration from real movies for the purposes of clarification.<br /